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Activity of any neuron with delayed feedback 
stimulated with Poisson stream is non-Markov 

Alexander K.Vidybida * 


Abstract 

For a class of excitatory spiking neuron models with delayed feed¬ 
back fed with a Poisson stochastic process, it is proven that the stream 
of output iuterspike intervals cannot be presented as a Markov process 
of any order. 
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1 Introduction 

Statistics of neuronal activity is often described as a renewal point process, or 
even a Poisson process, see [I] and references therein. On the other hand, in 
some sets of experimental data correlations are observed between consecutive 
interspike intervals (ISI), [2. 8,0, 5], which does not conform with the renewal 
hypothesis. What could be the reason of such correlations? In principle, 
any sort of memory in the neuronal firing mechanism could bring about 
memory into the sequence of ISIs, thus disrupting a possibility for it to be 
renewal. Memory in the bring mechanism can appear due to partial reset of 
the membrane potential after bring, PE], or due to threshold fatigue [8], or 
for other reasons, see pj for a review. 

Biologically, non-renewal statistics of neuronal activity can improve dis¬ 
crimination of weak signals Pi and therefore is essential feature of func¬ 
tioning of a nervous system. In this context, it was checked in [3] if it is 
possible to represent activity of electrosensory neuron as a Markov chain of 
some bnite order. Conclusion made in |3j is that the corresponding order, if 
any, cannot be lower than 7. 
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Normally, any neuron is embedded into a network. Inter-neuronal com¬ 
munication in the network is delayed due to finite speed of nervous impulses. 
In a reverberating network, this brings about one more reason for non-renewal 
firing statistics — the delayed feedback. We study here the simplest possible 
case of a network — a single neuron with delayed feedback. In the previous 
paper |10j . it was proven for a concrete neuronal model — the binding neu¬ 
ron with threshold 2 — stimulated with Poisson stream of input impulses, 
that statistics of its ISIs is essentially non-Markov. In this paper, we refine 
and extend methods of ro making those applicable to any neuron, which 
satisfies a number of very simple and natural conditions (see Cond0-Cond4 
in n. 2.1). Under those conditions, we prove rigorously that ISI statistics of 


a neuron with delayed feedback cannot be represented as a Markov chain of 
any finite order. 


2 Definitions and assumptions 

2.1 Neuron without feedback 

We do not specify any concrete neuronal model, only expect that a neuron 
satisfies the following conditions: 

• CondO: Neuron is deterministic: Identical stimuli elicit identical spike 
trains from the same neuron. 

• Condi: Neuron is stimulated with input Poisson stream of excitatory 
impulses. The input stream has intensity A. 

• Cond2: Neuron may fire a spike only at a moment when it receives an 
input impulse. 

• Cond3: Just after firing, neuron appears in its standard state, which is 
always the same. 

• Cond4: The output interspike interval (ISI) distribution is character¬ 
ized with a probability density function (pdf) p°(t), which is positive: 

t > 0 p°(t) > 0, and bounded: supp°(f) < oo. 

t> o 

The CondO, above, is imposed in accordance with experimental observa¬ 
tions, see e.g. mum As regards the Condi, Poisson stream is a standard 
stimulation when neuronal random activity is studied. The Cond2, above, 
is satisfied for most threshold-type neuronal models, starting from standard 
leaky integrate and fire (LIF) neuron |_13j and its modifications, see pRE]. In 


2 



order the Cond2 to be valid, it is enough that the following three conditions 
are satisfied: (i) neuronal excitatiorQ gets abrupt increase at the moment 
of receiving input impulse^ (ii) after that moment, the degree of excitation 
does not increase (it decreases for most neuronal models) until the next input 
impulse, (iii) the neuron fires when its degree of excitation exceeds a thresh¬ 
old level. The threshold can be either static, as in the basic LIF model, 
or dynamic [15]. These conditions seem to be standard for many threshold 
neuronal models used, see p ms HZ! and citations therein. Cond3 means 
that any kind of memory about previous input/output activity, which can 
be present in a neuron, is cleared after each triggering. Due to Cond3, out¬ 
put stream of neuron without feedback will be a renewal stochastic process. 
Cond4 seems to be natural for any neuronal model stimulated with Poisson 
stream. At least, all the five conditions are satisfied for the binding neuron 
model and for the basic LIF model, see PS, IS!, where p°(t) is calculated 
exactly for each model, respectively. 

2.2 Feedback line action 

We expect that each output impulse fired by neuron is fed back to the neu¬ 
ron’s input through a feedback line. The feedback line has the following 
properties: 

• Propl: The time delay in the line is A > 0. 

• Prop2: The line is able to convey no more than one impulse. 

• Prop3: The impulse conveyed to the neuronal input is identical to that 
from the input Poisson stream. 

ft is known that a neuron can form synapses (autapses) on its own body, 
or dendritic tree, e.g. [201 El]. This substantiates consideration of a sin¬ 
gle neuron with feedback not only as the simplest reverberating ’’network” 
possible, bat also as an independent biologically relevant case. The delay A 
comprises the time required by the output spike to pass the distance from 
axonal hillock, where it is generated, to the autapse and the synaptic de¬ 
lay. The Prop2 is somehow related to the refractoriness even if we do not 
introduce here the refractoriness to its full extent. The Prop3 means that we 
consider here an excitatory neuron. 

1 We use here term “excitation” instead of “depolarization voltage” because we do 
not specify any triggering mechanism. Our consideration as regards feedback shaping of 
firing statistics could be valid also for essentially artificial neurons, where excitation not 
necessarily has a voltaic nature. 

2 If considering an input impulse as a current impulse, then it has a 5-function form. 
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The important for us consequence of Prop2 is that at any moment of 
time the feedback line is either empty, or conveys a single impulse. If it does 
convey an impulse, then its state can be described with a stochastic variable 
s, which we call further “time to live”. The variable s denotes the exact 
time required by the impulse to reach the output end of the line, which is 
the neuron’s input, and to leave the line, ft is clear that 0 < s < A. In what 
follows, we use the time to live s only at moments when an ISI starts (just 
after triggering). 

Now it is worth to notice that each triggering starts a new ISI. And at 
the beginning of any ISI the line is never empty, but holds an impulse. This 
happens for the following reasons: 

a) If neuron is triggered by an impulse from the Poisson input stream, and 
the line was empty just before that moment, then the emitted impulse 
enters the line. At that moment the line is characterized with s — A. 

b) If neuron is triggered by an impulse from the Poisson input stream, 
and the line already conveys an impulse at that moment with time to 
live s, then that same impulse with that same time to live is retained 
at the beginning of the ISI that starts after that triggering, and the 
line is characterized with that same s. 

c) If neuron is triggered by an impulse from the line, then the line is empty 
at the firing moment and the emitted impulse enters the line. After 
that moment the line is characterized with s = A. 

2.3 Proof outline 

We expect that defined in nn. |2.1[|2.2| system of neuron with delayed feedback 
line fed with Poisson stream is in its stationary regime. This can be achieved 
if the system functions long enough that its initial state is forgotten. 

In the stationary regime, let p(t n ,..., ti) denotes the joint probability 
density function of neuron with delayed feedback. The probability to get, 
in the output, starting from the beginning, n consecutive ISIs 
such that t\ € [f; t l + dt t [, i = 1 with infinitesimal dti is given by 

p{tn, ■ ■ -,ti)dti... dt n . 

Let p(t n+ 1 | , t 0 )dt n+ i denotes the conditional probability to get the 

duration of (n + 2)-th ISI in [t n+ 1 ; i n +i + dt n+ \ [ provided that previous n + 1 
ISIs had duration t n ,... ,t 0 , respectively. 

Now we reformulate in terms of probability density functions the defini¬ 
tion from p2j, Cli.2 §6]: 
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Definition 1. The sequence of random variables {tj}, taking values in ft, is 
called the Markov chain of the order n > 0, if 



and this equation does not hold for any n' < n. 

In particular, taking m — n + 1, we have the necessary condition 

P(tn +1 | t n , ■ ■ ■ j Hi t 0 ) P(tn +1 I t n , . . . , ti), ti £ 12, i 0, . . . , Tl T 1, (1) 

required for the stochastic process {tj} to be n-order Markov chain. In the 
case of ISIs one reads ft = M + . 

We intend to prove that the relation ([TJ) does not hold for any n. For this 
purpose we calculate exact expression for p(t n+ i | t n ,..., t 0 ) as 


P(tn+ 1 ) t n , . . . , to) 
Pifni ■ ■ ■ i to) 



( 2 ) 


from which it will be clearly seen that the t 0 -dependence in p(t n+ i | t n ,... , t 0 ) 
cannot be eliminated whatever large the n is. 

As it is seen from (J5J), we need initially to calculate exact expressions for 
p(t n ,..., to) with arbitrary n. In [10], for the binding neuron model with 
threshold 2 this is done by introducing an auxiliary stochastic process with 
events (£j,Sj), where s* is the time to live at the beginning of ISI tj. It was 
proven that the sequence of events (tj,Sj), i = 0,1,..., is Markov chain, 
which helps to calculate the joint probability density p((t n , s n ), ..., (to, s o)) 
and then p(t n ,..., t 0 ) as marginal probability by integrating it over ]0;A] 
with respect to each s t . To simplify this approach, it is worth to notice 
that in the sequence of consecutive random events (t n ,s n ),..., (t 0 ,s 0 ) on ly 
the values of variables t n ,..., t l5 1 0 , s 0 are fairly random. Indeed, with t 0 , s 0 
given, one can figure out exact value for the sp if to < so then si = so —1 0 , 
and si = A otherwise. Now, with ti,si known, the same way it is possible 
to find the exact value of S2 and so on. This allows one to reconstruct 
unambiguously all the values Si,...,s n from the given sequence of values 
of t n ,..., ti, t 0 , So- Having this in mind, we introduce the conditional joint 
probability density p(t n+ 1 ,..., to | s), which we use to calculate required joint 
pdfs as follows 


A 



( 3 ) 


o 
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where s (dented previously as so) is the time to live at the beginning of ISI to, 
f(s) is the stationary pdf which describes distribution of times to live at the 
beginning of any ISI in the stationary regime. In what follows we analyze the 
structure of functions f(s) and p(t n+ 1 ,... ,to | s). It appears that f(s ) has 
a singular component a5(s — A) with a > 0, and p(t n+ 1 ,..., to | s ) has a 5- 
function-type singularities at definite hyper-planes in the (n + 2)-dimensional 
space of its variables (t n+ 1 ,..., t 0 ). After integration in (|3]), some of those 5- 
functions will survive, and one of those survived has its argument depending 
on t 0 . The latter statement depends on exact value of ISIs in the sequence 
t n+ 1 ,..., to- Here, we limit our consideration to the domain in the (n + 2)- 
dimensional space of variables (t n+ 1 ,..., to), which is defined as follows 


^t, < A. 


(4) 


i =0 


Notice that t n+ \ is not involved in Q. 

The t 0 -dependent (5-function will as well survive in the p(t n+ 1 | t n ,..., t 0 ) 
for any n, which will complete the proof that the condition (|TJ) cannot be 
satisfied for any n. 

A question remains of whether the domain Q has a strictly positive 
probability. This indeed takes place due to positiveness of pdfs p(t n+ 1 ,..., to) 
for any positive values of (t n+1 ,... ,t 0 ). The latter follows from the exact 


expressions for p(t n+1 ,... ,t 0 ) given in n. 3.4.1, Eq. (19). 


3 The proof 


3.1 Structure of functions p(t n+ 1, ..., to | s) 

Expect that the inequality Q holds. In order to perform integration in (|3]) , 
we split the integration domain into the following n + 2 disjoint sub-domains: 


Du — 


k -1 




4 = 0 


i =0 


, /c = 0,... ,ra, D n+1 = 


4=0 


A 


It is clear that 


n+1 


=]<>; A], 

k =0 


The conditional pdf p(t n+ i ,..., t 0 \ s ) has different structure at different 
domains. If s € D^, then a relation between s and fj is as shown in Fig. [lj 
As it could be suggested by Fig. [lj the Erst k — 1 ISIs are produced with 
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to t\ tk t n 


Fig 1: Mutual disposition in time of s and ■ ■ • > t n if s G D 

the delay line not involved. The k -th ISI is generated with the line involved. 
The corresponding time to live is Sk = s — U < tk, the next time to live 
is Sfc + i = A. Therefore, the structure of p[t n+ 1, ..., t 0 | s) at D k is as follows 


P(tn+ 1) ■ ■ ■ j ^0 | P(tn+ 1; • • • , tk+ 1 | A) p 


tk 



k-1 


IJAU), (5) 


i =0 


where k — 0,1,..., n. And if s £ D n+l} then relation between s and t t is as 



to t\ tk t n 


Fig 2: Mutual disposition in time of s and to,...,t n if s G D n+ 
shown in Fig. [2j This suggests the following structure for p(t n+ 1 ,..., to | s) 

( n \ n 

tn+l | S - ^ti ) ]^[p 0 (tj), s G D n+ 1 . (6) 

i=0 / i=0 

Here p(t \ s ) denotes the conditional pdf to get ISI of duration t if at its 
beginning, time to live of impulse in the feedback line is s. 

By utilizing the same reasoning with Q taken into account, one can 
represent the first factor in (J5| as follows 

t n \ n 

tn+l I A - tj 1 p°(ti). (7) 

i=k -\-1 / i=k-\-l 

Representation of p{t n+ 1 ,..., f 0 | s) by means of p°(t) and p(t \ s ), similar 
to that displayed in ([5]), (|6]), ([7]), can be as well constructed if Q does not 
hold. For our purpose it is enough to have ([5]), ([6]) and ([7]). 
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3.2 Structure of function p(t \ s ) 

Expect that at the beginning of an ISI, there is an impulse in the feedback 
line with time to live s. Then the probability that this ISI will have its 
duration t < s does not depend on the feedback line presence. Therefore, 

t < s => p{t | s ) = p°(t). 

The probability to get exactly t = s is not zero, because in this case the 
impulse, which triggers the neuron and finishes the ISI under consideration 
comes from the delay line. In order this to happen, it is necessary and 
sufficient that the following two events take place: (i) the neuron does not 
Ere at the interval ]0;s[; (ii) at the moment s, the neuron, due to previous 
stimulation from the Poisson stream, achieves such a state that adding one 
more input impulse will trigger it. The probability of (i) and (ii) is 
which can be easily concluded from the definition of p°{t). Thus, 

JjP('t') 

le]s-e;s + e[=t p(t \ s ) = ——^-5(s — t) 

A 

with infinitesimal e > 0. If the neuron still not triggered at moment s, then 
it is triggered by an input impulse from the Poisson stream at t > s. The 
probability to get such an impulse in [t\ t + dt[ is Xdt. Therefore, one can 
expect that for t > s, p(t \ s) < A. 

Based on the above reasoning we represent p{t \ s ) in the following form 
P(t | s) = p\t | s) + ^-S(s-t), (8) 

where p b {t | s) is a bounded functiorj^J 

3.3 Structure of probability density function f(s) 

In the stationary regime, the pdf f(s) must satisfy the following equation 

A 

/0) = j p (s I s')f(s')ds', (9) 

0 

where the transition function P(s | s') gives the probability density to hnd 
at the beginning of an ISI an impulse in the line with time to live s provided 

3 Compare this with J23 Eq. (7)], where p(t \ s) is calculated exactly for the binding 
neuron model. 






at the beginning of the previous ISI, there was an impulse with time to live 
s'. 

To determine exact expression for P(s | s') we take into account that after 
single firing, time to live can either decrease, or become equal A. Therefore, 

s' <s< A => P(s | s’) = 0. (10) 

If s < s', then the firing, which causes transition from s' to s, happens 
without the line involved. Therefore, 

0 < s < s' =>• P(s | s')ds = p°(s' — s)ds. (11) 


Finally, it is possible that starting from s' one obtains s = A after the next 
firing. In order this to happen, it is necessary and sufficient that no firing 
happens during s' units of time. And this happens with probability 


s' 



0 


Having this in mind, one could conclude that in the plane (s, s'), at the 
straight line s = A, s' — any, the P(s | s') has singularity of the following 
form: 


PV)j(s - A). 


( 12 ) 


Now, with (10)-(12) taken into account, Eq. 0 can be rewritten as 
follows 


A A 

f(s) = j p°(s’ - s)f(s’)ds' + 5(s - A) J P°(s')f(s')ds'. 

s 0 

It is clear from this equation that f(s) has the following formQ 

f(s) = g(s) + a5(s - A), (13) 

where a > 0 and g(s) is bounded and vanishes out of interval ]0; A], 

4 Compare this with [2£ Eqs. (14)-(16)], where f(s) is calculated exactly for the binding 
neuron model. 
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3.4 Form of p(t n+ 1 ,..., to) and p(t n ,..., to) after integra¬ 
tion in 


Let D = (J Dk■ At D, representations (5) and (7) are valid. Also at D, f(s) 
k =o 

reduces to g(s). Therefore, 

I p(t n+ 1 ,.. . ,t 0 I s)f(s) = 

D 

n / n \ n /* / k—1 \ 

= ^P I t n+ 1 | A - L I JJ p°(ti) / p ( tfc | s - ) p(s)ds. (14) 

fc=0 V i=fc+l / i = 0 n V i=0 / 

L -'k 

i ^ k 


Taking into account (|8l) it can be concluded that expression (14), after per 


forming integration, does not have any term with 5-function depending on 
to- 

Consider now the remaining part of integral in (J3]). With i§ taken into 
account one has: 

/ p(tn+l,---,to\s)f(s)=f[p 0 {ti) f P ( t n+ l | S - ^Tti J f(s)ds. 

n - i=0 Dl +1 V *=0 J 


After substituting here expressions (|8|) , (13) one obtains four terms: 


P(t n + i, — ,| s)f(s) = 


Dn 


n r f n \ 

/ P I tn+l | S - J f(s)ds 

i=0 D n+1 V *=0 J 

n r { n \ 

II AU) / p b I t n+1 I s - J g(s)di 

1=0 dL V *=o J 


Y\_p°(ti)p b I t n+ 1 A - y tj 


i =0 


i=0 


n+l 


n+l 


n+l 


(SC**) +- x IIAU)6 () ■ (15) 

i=0 \ 1=0 J 1=0 \ i= o 


After performing integration, only the fourth term here includes a 5-function. 
And argument of this 5-function does depend on to. 
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After taking (14) and (15) together we conclude that the required joint 


probability density has the following form 


n+1 


n+1 


p(t n + 1, ■■■,to)= P w (t n+ 1, ■ ■ ■ ,t 0 ) + y (a - ^ti j , (16) 


i =0 


i=0 


where function p w (t n+ 1 ,... ,to) does not have singularities depending on f 0 . 

3.4.1 Form of p(t n ,..., f 0 ) after integration 

If Q is satisfied, then we have similarly to ([5]), (|6]) 

/ k -1 \ fc-1 

p(t n , ...,t 0 \s)= p(t n , . . . ,4+1 1 A) p 1 4 I s - tj | 

\ i =0 / i=0 

s G Hfc, k — 0,..., n — 1, 

t n—1 \ n—1 

4 1 s - J TTp°(4), s e D n . 

i=0 J i= 0 

Again due to (|4|), and in analogy with ([T]) we have instead of the last two 
equations the following one: 

t k—1 \ n 

4 1«- j n p 

»=o / i = o 
i ^ k 

s E Dk, k = 0, ...,n. (17) 

ft is clear that expression similar to Q turns here into the following 


p(t n ,... ,t 0 | s) = JJp°(4), 
Now, due to (|I7|), Q we have 


n+1 • 


(18) 


i=0 


P(tn, ■ ■ ■ , to) = J P(tn, • • • , 4 | S)f(s)ds = 
0 


n n r ( k—1 \ 

yi n p°(fi) / p (^ i s - ^) ^( s )^+ 

fc=0 j = o i. V i=0 / 


i^k 


n « 

n/(b) / /(s)ds. (19) 

i=0 rG 


■Dn + 1 
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3.5 ^-dependence cannot be eliminated in p(t, 


n +1 


t 


ni 


to) 


Now, with representations (16) for p(t n+ 1 ,... ,t 0 ) and (J19]) for p(t n ,... ,t 0 ) 
we can pose a question about the form of p(t n+ 1 | t n ,... ,t 0 ). The latter can 
be found as defined in (|2]). First of all notice that due to (19) and Cond4, 


p{t n ,... ,to) is strictly positive for positive ISIs. This allows us to use it as 
denominator in the definition (j2j). Second, it can be further concluded from 
(19) and Cond4, that p(t n ,..., t 0 ) is bounded, and therefore does not include 
any singularity of 5-function type. The latter means that any singularity 
contained in the p(t n+ 1 ,..., to) appears as well in the p(t n+1 | t n ,... ,t 0 ). It 
follows from the above that the conditional pdf p(t n+ 1 | t n ,... ,t 0 ) can be 
represented in the following form: 

/ n+l \ 

P(tn +1 | tni ■ ■ • > to) P (tn +1 | t n , . . . , to) T Q{t n -\- 1, • • • , tg)5 I A ^ ^ tj J , 


i =0 


( 20 ) 

where p w (t n+ \ | t n ,..., to) does not contain any 5-function depending on t 0 , 
and Q[t n+ 1 ,... ,t 0 ) is strictly positive bounded function: 


Q{tn+li ■ ■ ■ i to) 


n+l 

a n P°(ti) 

i =o _ 

A p(t n , - - - , t 0 ) ' 


The representation (20) thus proves unequivocally that for any n, conditional 


pdf p(t n+ 1 | t n ,... ,t 0 ) does depend on t 0 (the second term in (20)) and this 
dependence cannot be eliminated. 


4 Conclusions and Discussion 

We have proven here that any neuronal model, which satisfies Cond0-Cond4, 
above, and is equipped with a delayed feedback, will display essentially non- 
Markov activity expressed in terms of output ISIs, when stimulated with 
Poisson stream. This has a consequence for admissible approaches while 
modeling activity of neuronal networks with stochastic behavior. Indeed, in 
a reverberating network, a delayed feedback mediated by other neurons is 
always present. Our result suggests that in this case, activity of individual 
neurons in the network should be essentially non-Markov. Another situation 
in networks with instantaneous interneuronal communication. In the case of 
no delay communications, the neuronal activity can well be Markov, or even 
Poisson, see example in (24j. 
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We used here a single neuron with delayed feedback as the simplest case 
of reverberating ’'network”. At the same time, neurons which send to them¬ 
selves their output impulses are known in real nervous systems, [2D, 2Tj . 
Therefore, our conclusions about essentially non-Markov behavior should be 
valid for those neurons even without taking into account their involvement 
in a wider network activity. 

The set of conditions Cond0-Cond4 while being rather natural and wide 
enough, leaves out of our consideration many neuronal models known in neu¬ 
roscience. E.g., Cond2 excludes models with spike latency. Cond3 excludes 
models with internal memory extending beyond a single ISI duration. Thus, 
we do not consider here partial afterspike resetting m, threshold fatigue 
[8], another types of adaptation, like multi-timescale adaptive threshold [25j . 
Any kind of adaptation in individual neuron is by itself able to bring about a 
kind of memory in the neuronal output stream. Therefore, considering neu¬ 
rons without adaptation we demonstrate here, that delayed feedback without 
additional memory-like mechanisms known for neurons makes neuronal out¬ 
put essentially non-Markov. 

Another limitation is Condi — we use a Poisson process as a stimulus. 
It seems that the proof given here can be extended to a wide class of renewal 
processes taken as stimuli. This will be checked in further work. 
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